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Abstract 

This paper generalizes the notion of stochastic order to a relation 
between probability measures over arbitrary measurable spaces. This 
generalization is motivated by the observation that for the stochas- 
tic ordering of two stationary Markov processes, it suffices that the 
generators of the processes preserve some, not necessarily reflexive or 
transitive, subrelation of the order relation. The main contributions 
of the paper are: a functional characterization of stochastic relations, 
necessary and sufficient conditions for the preservation of stochastic 
relations, and an algorithm for finding subrelations preserved by prob- 
ability kernels. The theory is illustrated with applications to hidden 
Markov processes, population processes, and queueing systems. 
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1 Introduction 

Comparison techniques based on stochastic orders [21, 23, 26] are key to 
obtaining upper and lower bounds for complicated random variables and 
processes in terms of simpler random elements. Consider for example two 
ergodic discrete-time Markov processes X and Y with stationary distribu- 
tions fix and /xy , taking values in a common ordered state space, and denote 
by <st the corresponding stochastic order. Then the upper bound 

<st ^^Y (1) 
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can be established [12] without explicit knowledge of fix by verifying that 
the corresponding transition probability kernels Px and Py satisfy 

x<y ^ Pxix,-) <stPY{y,-). (2) 

Analogous conditions for continuous-time Markov processes on countable 
spaces have been derived by Whitt [29] and Massey [20], and later extended 
to more general jump processes by Brandt and Last [3]. 

The starting point of this paper is to generalize the notion of stochastic 
order by denoting X ~st Y, if there exists a coupling {X, Y) of X and Y 
such that X ~ y almost surely, where ~ denotes some relation between the 
state spaces of X and Y. The main motivation for this definition is that (2) 
is by no means necessary for (1); a less stringent sufficient condition is that 

x ~ y ^ Px{x,-) '^st Priy,-) (3) 

for some, not necessarily symmetric or transitive, nontrivial subrelation of 
the underlying order relation. Another advantage of the generalized defi- 
nition is that X and Y are no longer required to take values in the same 
state space, leading to greater fiexibility in the search for bounding random 
elements Y. For example, to study whether f{X) <st g{Y) for some given 
real functions / and g defined on the state spaces of X and Y, we may define 
a relation x ~ y by the condition /(x) < g{y) [5]. 

The main contributions of the paper are: a functional characterization of 
stochastic relations, necessary and sufficient conditions for the preservation 
of stochastic relations in the sense of (3), and an algorithm for finding sub- 
relations preserved by probability kernels. The functional characterization 
(Section 2) is given in terms of relational conjugates that were implicitly de- 
fined by Strassen [25, Theorem 11], and the proof goes along similar lines, the 
new feature being the use of compact sets and upper semicontinuous func- 
tions instead of completions of measures. Lopez and Sanz have characterized 
the preservation of stochastic relations for Markov processes on countable 
spaces in terms of a subtle order construction [18]. Section 3 describes an 
equivalent, considerably simpler characterization based on relational conju- 
gates, together with an iterative algorithm for finding the maximal subrela- 
tion of a given relation preserved by a pair of probability kernels. The main 
results are extended to the context of general random processes and Markov 
processes in Section 4. Applications to hidden Markov processes, popula- 
tion processes, and queueing systems are discussed in Section 5. Section 6 
concludes the paper. 
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2 Stochastic relations 



2.1 Definitions 

Let {Si, Si) and [82,32) be measurable spaces, and denote by ViSi) the 
family of probability measures on (Si, Si). Unless otherwise mentioned, all 
spaces shall implicitly be assumed Polish (complete separable metrizable) 
and equipped with the Borel sigma-algebra. A coupling of probability mea- 
sures fii € T^iSi) and fj.2 G 'P{S2) is a probability measure n G V{Si x 5*2) 
with marginals fii and fi2, that is, /iovr"^ = /ij for i = 1,2, where TTj denotes 
the projection map from Si x 52 onto Si. If is a coupling of /Ui and H2, 
we also say that fi couples fii and ^2 [16, 27]. 

A measurable relation between 5i and ^2 is measurable subset of x 
S2. All relations in this paper are assumed to be closed (in the product 
topology of Si X S2), if not otherwise mentioned. Given a nontrivial {R 7^ 0) 
measurable relation R between Si and S2, we write xi ~ X2, if (2:1, X2) G R. 
For probability measures ni E ViSi) and iJ,2 € 'P{S2) we denote 

/il ~st /^2, 

and say that /xi is stochastically related to fj,2, if there exists a coupling // 
of /.ii and /i2 such that fi{R) = 1. The relation Rgt = {(/Ui,/i2) : /"i ~st ^^2} 
is called the stochastic relation generated by R. Observe that two Dirac 
measures satisfy 6x1 ~st if ^^'^ only if xi ^ X2. In this way the stochastic 
relation i?st may be regarded as a natural randomization of the underlying 
relation R. 

A random variable Xi is stochastically related to a random variable X2, 
denoted by Xi ~st X2, if the distribution of Xi is stochastically related to 
the distribution of X2. Observe that Xi and X2 do not need to be defined 
on the same probability space. Recall that a coupling of random variables 
Xi and X2 is a bivariate random variable whose distribution couples the 
distributions of Xi and X2. Hence Xi ~st X2 if and only if there exists a 
coupling (Xi,X2) of Xi and X2 such that Xi ^ X2 almost surely. 

Example 2.1 (Stochastic equality). The stochastic relation generated by 
the equality relation {{x,y) : x = y} on 5 is the equality on V(S). Hence 
X =st Y if and only if X and Y have the same distribution. 

Example 2.2 (Stochastic e-distance). Define a relation on the real line by 
denoting a; ~ y, if |x — y| < e. If Xi ~st X2, then the cumulative distribution 
functions of Xi and X2 satisfy 

F2{x - e) < Fi{x) < F2{x + e) for ah x. (4) 
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Conversely, if (4) holds, it is not hard to verify that the quantile functions 
Gi(r) = inf{x : Fi{x) > r] satisfy \Gi{r) - G2{r)\ < e for all r G (0,1). 
Hence the bivariate random variable X = (Gi(^), G2(^)), with ^ uniformly 
distributed on (0, 1), couples Xi and X2 and satisfies Xi ^ X2 with proba- 
bility one. Thus (4) is necessary and sufficient for Xi ~st X2. 

Example 2.3 (Stochastic majorization). Let S be closed subset of M" , and 
denote by x^ij > • • • > xj^j the components of x € S* in decreasing order. The 
weak majorization order on S is defined by denoting x y, if ^(L;^ xjj] < 
Yli=i y[i] fo'^ all /c = 1, . . . , n; and the majorization order by denoting x ^™ 
y, if x y and Y17=i-'^i ~ X^ILi?/*- The ^™-increasing real functions 

are called Schur-convex, and a function is ^ ""^-increasing if and only if it 
is coordinatewise increasing and Schur-convex [19, Theorem 3. A. 8]. The 
standard characterization for stochastic orders (Remark 2.6 in Section 2.3) 
hence shows that X Y (resp. X Y) if and only if E f{X) < E f{Y) 
for all positive measurable Schur-convex (resp. coordinatewise increasing 
Schur-convex) functions /. 

2.2 Relational conjugates 

To develop a convenient way to check whether two probability measures are 
stochastically related or not, we shall define the right conjugate of Bi C Si 
and the left conjugate of B2 C ^2 with respect to a relation R by 

B{* = Ux(^BAy e 52 : X ~ y}, 
B^ = UyeB^I^; G Si : X ~ y}. 

The conjugates of positive functions on Si are defined analogously by 

= sup /i(x), y G 52, 

xGSi'.x^y 

ff{x) = sup /2(y), X G 5i, 

where we adopt the convention that the supremum of the empty set is zero. 
Relational conjugates of sets and functions are interlinked via 

{iB,r = iB^, (5) 

where Ib^ denotes the indicator function of Bi, and 

{x : /i(x) > r}^ = {y : /f (y) > r}, (6) 



4 



which is vahd for all r > 0. 

The following result summarizes the basic topological properties of right 
conjugates. By symmetry, analogous results are valid for left conjugates. 

Lemma 2.4. Let R be a closed relation between two Polish spaces. Then: 

(i) B~* is closed for compact B. Especially, {x}~* is closed for all x. 

r^^j ^-s UPP^''^ semicontinuous (u.s.c.) for all positive u.s.c. f on Si with 
compact support. 

Proof. Assume B is compact, and consider a sequence Un ^ y such that 
yn G B^ for all n. Then for all n there exists Xn ^ B such that Xn ^ y-n- 
Because B is compact, there exists x (z B such that Xn ^ x as n ^ oo 
along some subsequence of the natural numbers. Hence y„) {x,y) as 
n ^ oo along the same subsequence, which implies that x ~ y, and thus 
yeB^. 

Assume next that / is positive u.s.c. with compact support on Si. We 
shall first show that 

{x : f{x) > r}^ = {y : f^{y) > r} for all r > 0. (7) 

Observe first that if y belongs to the left side of (7), then f{x) > r for some 
X ~ y, so that f^{y) > r. To prove the converse statement, assume next 
that f^{y) > r. Then the sets Kn = {/>?' — 1/?^} H {y}*~ are nonempty 
and compact for all n > 1/r, because {y}*~ is closed by property (i). Hence 
Cantor's intersection theorem implies that 

{x : f{x) > r} n {y}"^ = r\n>i/rKn, 

is nonempty, so that /(x) > r for some x ~ y. We may now use (7) together 
with property (i) to conclude that {y : /^(y) > r} is closed for all r > 0. 
Obviously, {y : f^{y) > 0} = 52 is closed as well. □ 

2.3 Functional characterization 

The following result characterizes stochastic relations using relational conju- 
gates of sets and functions. The key part of the characterization is essentially 
Strassen's Theorem 11 [25], written in a new notation. The new contribu- 
tions are (ii) and (iv), providing classes of test sets and functions with Borel- 
measurable conjugates (Lemma 2.4) that are large enough to characterize 
stochastic relations without resorting to completions of measures. 
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Theorem 2.5. Let R be a closed relation between Polish spaces Si and 82- 
Then fi -^st i' is equivalent to each of the following: 

(i) n{B) < v{B^) for all measurable B such that B^ is measurable. 

(a) n{B) < v{B~*) for all compact B. 

(Hi) f dfi < f^ dv for all positive measurable f such that f^ is 
measurable. 

r^'^-^ Isi f — Is2 '^^ positive u.s.c. f with compact support. 

Remark 2.6. If R is an order (reflexive and transitive) relation on S, then 
using the properties B C B^ = (B^)^ and / < = (/^)^ we see that 
(i) and (iii) in Theorem 2.5 become equivalent to well-known characteriza- 
tions of stochastic orders [12, 25]: 

(i') fJ^{B) < i^{B) for all measurable upper sets B. 

(iii') Jg f df-i < f diy for all measurable positive increasing functions /. 

Remark 2.7. When 5i and «S'2 are countable, the measurability require- 
ments of Theorem 2.5 become void, and the word "compact" becomes re- 
placed by "finite". 

Proof of Theorem 2.5. fi ~st =^ (i)- Let A be a coupling of fi and v 
such that X{R) = 1. Then because 

{B xS2)nR={B X B^) n R, 

we see that 

1^l{B) = \{B X ^2) = X{B X B-^) < X{Si X S"") = j^(5"") 

for all measurable B C Si such that B^ is measurable. 

(i) =^ (ii). Clear by Lemma 2.4. 

(ii) (iv). Let / be a positive compactly supported u.s.c. function 
on Si. Then equality (7) shows that 

lj{{x : f{x) > r}) < u{{y : f^iy) > r}) 

for all r > 0. The validity of (iv) hence follows by integrating both sides of 
the above inequality with respect to r over (0, 00). 
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(iv) =^ --^st By virtue of [25, Theorem 7], it suffices to show that 
/ fd^i+f gdv< sup (8) 

JSi JS2 {x,y)eR 

for all bounded continuous / and g on and ^2, respectively, and without 
loss of generality we may assume / and g are positive and bounded by one. 
Given any such functions / and g, and a number e > 0, choose a compact 
set K C Si such that n{K^) < e [1, Theorem 1.3], and define /o = /Ik- 
Because /o is u.s.c, we see using (iv) that f^^ /o dfi < f^^ di^, so that 

/ fd^i+ [ gdv<j {f^+g)dv + e. (9) 

In light of (5), assumption (iv) further implies that ^l[K) < u{K^), because 
Ik is u.s.c. Thus v{[K'^Y) < e, so by splitting the i^-integral into K~* and 
its complement we see that 

/ (/(T + 9)di^< sup if^iy) + giv)) + 2e. 
J y&K^ 

Because /^p < and C , the above inequality combined with (9) 
shows that 

/ / gdv< sup (/^(2/)+5(2/))+3e. 

JSi JS2 j/eV 

After letting e ^ and observing that 

sup {n{y)+g{y)) = sup (/(x) +g{y)), 

y&S^ {x,y)eR 

we may conclude that (8) holds. 

Finally, observe that the proof of (i) =^ (iii) is completely analogous 
to the proof of (ii) =^ (iv), and the implication (iii) =^ (iv) follows 
immediately by Lemma 2.4. □ 



3 Preservation of stochastic relations 
3.1 Coupling of probability kernels 

Monotone functions are key objects in the study of order relations. When 
passing from orders to general relations, the role of monotone functions is 
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taken over by function pairs (/i, /2) such that xi ~ X2 =^ fi{xi) ~ f2{x2)- 
To study stochastic relations, we need a randomized version of the above 
property. Recall that a probability kernel from a measurable space S to a 
measurable space S' is a mapping P : S x S' ^ M. such that P{x, •) is a 
probability measure for all x, and x i-^^ P{x,B) is measurable for all i? € 5'. 
Probability kernels may alternatively be viewed as mappings 7^(5") B n 
HP e V{S') by defining ijlP{B) = f^P{x,B) ^i{dx). 

Given a closed relation R between Polish spaces 5i and 52, and probabil- 
ity kernels Pi on and P2 on 52, we say that the pair (Pi, P2) stochastically 
preserves R, if any of the equivalent conditions in Theorem 3.1 holds. 

Theorem 3.1. The following are equivalent: 
(i) xi ~ X2 ^> Pi(xi, •) ~st P2(a:;2, •)• 

(a) m ~st =^ Ail Pi ~st M2P2- 

(Hi) Pi(xi,P) < P2(x2,P~*) for all xi ~ X2 and compact B C Si. 

Proof, (i) =^ (ii). Given ^1 ~st A*2) choose a coupling of //i and /i2 such 
that /i(P) = 1. Theorem 2.5 then shows that 

/XiPi(P)= / P1(X1,P)/X(C?X) < / P2(x2,P^)^((ix) =/i2P2(P^) 
J_R J R 

for all compact P C Si, so that /UiPi ~st ^2P2- 

The implication (ii) =^ (i) follows immediately by choosing = dx^, 
while the equivalence (i) <^=^ (iii) is clear by Theorem 2.5. □ 

Remark 3.2. A probability kernel P is said to stochastically preserve a 
relation R on S, if xi X2 =^ P(xi,-) ~st P(3;2r)- Order-preserving 
probability kernels are usually called monotone [21]. 

The main result of this section is the following coupling characterization 
of relation-preserving pairs of probability kernels. For technical reasons 
related to local uniformization of Markov jump processes in Section 4.3, we 
shall consider probability kernels Pj from Si to S'^, where S'^ is a measurable 
space not necessarily equal to 5^. A probability kernel P from Si x ^2 to 
S'l X 52 is called a coupling of probability kernels Pi and P2, if the probability 
measure P(x,-) couples the probability measures Pi(xi,-) and P2(x2,-) for 
all X = (xi,X2) € 5i X 52- 
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Theorem 3.3. Given closed relations R between Si and S2, and R' between 
S'l and S'2, assume that 

Then there exists a coupling P of Pi and P2 such that P{x, R') = 1 for all 
X € R. 

The proof of Theorem 3.3 requires some prehminaries on topology and 
measure theory that are discussed next. Denote by vTj the projection from 
S'l X S'2 to Si, and define the projection maps vTj : P(Si x S2) T^iSi) 
by VTj^ = /i o Trj^"*^, so that VTj^ equals the i-th marginal of /i. From now 
on, all sets of probability measures shall be considered as topological spaces 
equipped with the weak topology. 

Lemma 3.4. The projection Tfi : V{Si x S2) 'PiSi) is continuous and 
open with respect to the weak topology, i = 1,2. 

Proof. Assume that fi"' ^ fiin V{Si x S2), and let / G Cb{Si). Then because 
/oTTj G Cfe(SixS2), it follows that vrj^"(/) = /i"(/o7rj) ^ ^(/ovTj) = 7rj^(/). 
Hence tTj is continuous. The openness of ifi follows from Eifler [s. Theorem 
2.5], because the map vTj is continuous, open, and onto. □ 

Lemma 3.5. For any fj.i € V{Si) and ^2 € 'P{S2), the set K{^i,^2) of all 
couplings of fii and ^12 is compact in the weak topology ofV{Si x S2). 

Proof. Given e > 0, choose compacts sets Cj C Si such that fJ-i{C^) < e/2, 
i = l,2. Define C = Ci x C2. Then because = (fll x S2) U (Si x C7^), it 
follows that 

MC^) </ii(Ci^)+^2(C2^) <e 

for all [I € K{fj,i,H2)- Hence K{^i,^2) is relatively compact by Prohorov's 
theorem. The equality K{ni,H2) = vrj"^(^i) n-7r^^(/X2) further shows that 
K{ni,iJi2) is closed, because vr, are continuous by Lemma 3.4. □ 

Lemma 3.6. Let P be a probability kernel from S to S' . Then the map 
X ^ P{x, •) is B{S) /B{V{S'))-measurable, where B{V{S')) denotes the Borel 
a-algebra generated by the weak topology on V{S'). 

Proof. For any / G Ch{S'), one may check by approximating / with simple 
functions that the map x 1— > P{x, f) is measurable. Hence it follows that the 
set {x : P{x, •) G A] is jB(S)-measurable for any A = n^=-^{;U : /u(/fc) G Bk}, 
where fk G Ci,{S') and are open subsets of the real line. Because the 
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sets of the above type form a basis for the weak topology of V{S'), and 
because the space V{S') equipped with the weak topology is Polish [1] and 
hence Lindelof, it follows that any open set in V{S') can be represented as 
a countable union of the basis sets. Hence {x : P{x, ■) G A\ is measurable 
for all open subsets A in V{S')^ and the claim follows. □ 

A set-valued mapping from a set S to a set S' is a function that assigns to 
each element in S a subset of S". A set-valued mapping F from a measurable 
space 5 to a topological space S' is measurable [28], if the inverse image 

F-[A) = {xeS: F{x)r\A^ 0} 

is measurable for all closed A d S' . 

Lemma 3.7. Let Pi he probability kernels from Si to S'^, i = 1,2. Then the 
set-valued mapping F : x ^ K{P{xi, •), P2{x2, •)) is measurable. 

Proof. Because the set F{x) C V{S[ x 6*2) is compact for all x by Lemma 3.5, 
it is sufficient to verify that F~(A) is measurable for all open sets A (Him- 
melberg [1(1, Theorem 3.1]). Let us hence assume that A C V{S[ x 5*2) is 
open. Observe that F~{A) = ttY^{Bi) Pi tt2^{B2), where 

= {xi € Si : Pi{xi, •) G TTiiA)}. 

Now Lemma 3.4 implies that 'n'i{A) is open, and Lemma 3.6 further shows 
that Bi is measurable. Thus F~{A) is measurable. □ 

Proof of Theorem 3.3. Assume without loss of generality that R' 7^ 0, and 
let V{R) = {fJ- e V{S[ X S'2) : = 1}. Because R' is closed, it follows 

from Portmanteau's theorem that V{R') is closed. Define the set-valued 
mappings F and G from 5ix52 to ViS'^xS'^) by F{x) = K{Pi{xi, ■),P2{x2, •)), 
and 

[ F{x), else. 

Then for any closed A' C V{S[ x 5^), 

G-{A') = {RnF~{V{R')nA')) U {R'' n F- {A')) . 

Because V{R') is closed, it follows from Lemma 3.7 that G~{A') is measur- 
able. Moreover, because F is compact-valued by Lemma 3.5, we may con- 
clude that G is a measurable set-valued mapping such that G{x) is compact 
and nonempty for all A measurable selection theorem of Kuratowski and 
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Ryll-Nardzewski [15] (see alternatively Srivastava [24, Theorem 5.2.1]) now 
shows that there exists a measurable function g : Si x S2 ^ V{S'i x S2) such 
that g{x) G G{x) for aU x. By defining P{x,B) = [g{x)]{B) for x e Si x S2 
and measurable B <Z S'l x S2, we see that P is a probability kernel from 
5i X ^2 to S[ X S2 with the desired properties. □ 

3.2 Subrelation algorithm 

This section presents an algorithm for finding the maximal subrelation of a 
closed relation that is stochastically preserved by a pair (Pi,P2) of contin- 
uous^ probability kernels. Given a closed relation R and continuous prob- 
ability kernels Pi on Si, i = 1,2, define recursively the relations -R^"^ by 
= R, 

= {xe : {Piixi, ■),P2{X2, •)) G , 

and denote 

00 

R* = f] (10) 

n=0 

Lemma 3.10 below shows that the relations i?^") are closed and hence mea- 

(n) 

surable, so the stochastic relations R^^ are well-defined. The following 
theorem underlines the key role of R* in characterizing the existence of 
subrelations stochastically preserved by a pair of probability kernels. 

Theorem 3.8. Assume Pi and P2 are continuous. Then R* is the maximal 
closed subrelation of R that is stochastically preserved by (Pi,i-*2)- Espe- 
cially, there exists a nontrivial closed subrelation stochastically preserved by 
(Pi, P2) if and only ifR*^$. 

The following three lemmas summarize the topological preliminaries re- 
quired for the proof of Theorem 3.8. 

Lemma 3.9. Let R be a closed relation between Polish spaces Si and S2- 
Then the relation R^t is closed in the weak topology ofV{Si) x V{S2)- 

Proof. Assume ^ /i and z/„ ^ z/ such that Hn ~st for all n. Then 
for all n there exists a coupling A„ of and Vn such that Xn{R) = 1- 
Because the sequences Hn and are tight, and because An((Ci x 6*2)'^) < 

probability kernel P from S to S' is called continuous if P{xn, •) — > P{x, •) in 
distribution whenever Xn ^ x. In other words, P is continuous if and only if the map 
X K-> P{x, •) from S to Vi^S') is continuous, when 'P{S') is equipped with the weak topology. 
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fj,n{Cf) + t'„(C|) for all compact Ci and C2, it follows that the sequence 
is tight, so there exists A G V{Si x 52) such that A„ ^ A as n — > 00 along 
some subsequence [11, Theorem 16.3]. The continuity of ttj (Lemma 3.4) 
implies that A is a coupling of /i and i/, and Portmanteau's theorem shows 
that X{R) = 1. Hence fi ~st ^- D 

Lemma 3.10. Given continuous probability kernels Pi and P2, define 

M{R) = {x£R: (Pi(xi,-),P2(X2,-)) e ^st} (H) 
for measurable relations R. Then: 
(i) M{R) C M{R') for R C R' . 
(a) M maps closed relations into closed relations. 

Proof. For (i) it suffices to observe that R C R' implies Rst C -Rgf For (ii), 
observe that M{R) = R Ci f~^{Rst), where the function f : Si x S2 ^ 
V{Si) X V{S2) is defined by f{x) = {Pi{xi, ■), P2{x2, •)). Because R^t is 
closed (Lemma 3.9) and / is continuous, it follows that M{R) is closed. □ 

Lemma 3.11. Let R^^^ D R^^^ D ■ ■ ■ be closed relations between Polish 
spaces Si and S2, and let R* = n^^QR^^h Assume that {111,122) € R^^^ for 
all n. Then {^1,^12) S -Rgf 

Proof. By definition, for all n there exists a coupling A„ of /.t and v such 
that \n{R^^^) = 1. Because the set of couplings of ^ and v is compact 
by Lemma 3.5, there exists a coupling A of /u and v such that An — ^ A 
as n ^ 00 along a subsequence of Z+. Further, observe that \n{R^^^) > 
An(i?('^)) = 1 for all m < n, which implies that limn— > 00 An(i?(™)) = 1 for 
all m. Portmanteau's theorem now shows that A(i?('")) = 1 for all m, so it 
follows that X{R*) = 1. □ 

Proof of Theorem 3.8. Let M be the map defined in (11). If x S i?*, then 
X e = M(i?(")) shows that {Pi{xi, •),P2(x2, •)) ^ ii^^ for all n. Be- 

cause the relations R^'^^ are closed by Lemma 3.10, we see using Lemma 3.11 
that {Pi{xi, P2{x2, ■)) £ Rtf Hence (^1,^2) stochastically preserves R* . 
On the other hand, if R' is a closed subrelation of R that is stochastically 
preserved by (^1,^2), then R' = M{R') C M{R) = i?^^) by Lemma 3.10. 
Induction shows that R' C i?^"^ for ah n, and thus R' C R*. □ 
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4 Random processes 



4.1 Random sequences 

Given a relation R between Si and S2, the coordinatewise relation between 
the product spaces and ^2 for n < cx3 is defined by 



The stochastic relation generated by i?" is called the stochastic coordinate- 
wise relation. The following example shows that the stochastic coordinate- 
wise relation of two random sequences cannot be verified just by looking at 
the one-dimensional marginal distributions. 

Example 4.1. Let ^1 and .^2 be independent random variables uniformly 
distributed on the unit interval, and define X = aiid Y = (^1,^2)- 

Then Xi =st Yi for all i = 1,2, but X and Y are not related with respect to 
the stochastic coordinatewise equality. 

The proof of the following result is a straightforward modification of its 
continuous-time analogue Theorem 4.6, and shall hence be omitted. 

Theorem 4.2. Two random sequences X and Y satisfy X r^st Y if and 
only if {Xtj^ , ■ ■ ■ , Xtf, ) ~st (^1 , ■ ■ ■ , Yt^. ) for all finite parameter combinations 



The following result, which is completely analogous to [12, Proposi- 
tion 1], gives a sufficient condition for X ~st Y in terms of conditional 
probabilities. Let Pi be a probability kernel from S\~^ to Si representing 
the regular conditional distribution of Xi given {Xi, . . . , and define 

the kernels Qi in a similar way for y [11, Theorem 6.3]. 

Theorem 4.3. Assume that Xi ~st Yi, and that for all i, 

Pi{xi, . . . ,Xi-i,dxi) ~st Qiiui, ■ ■ ■ ,yi~i,dyi) 

whenever {xi, . . . ,Xi-i) ~ {yi, . . . ,yi~i). Then X ~st Y. 

Proof. Let Ai be a coupling of the distributions of Xi and Yi such that 
Xi{R) = 1. For convenience, we shall use as a shorthand for (xi, . . . , 
By Theorem 3.3 there exists for each i a coupling Ai of probability kernels 
Pi and Qi such that Ai{{x'^~^ ,y'^~^), R) = 1 whenever ^ y*^^- Then it 
is easy to verify by induction that the probability measure 



i?" = {{x, y) £ S"^ X S^ -.Xi^ yi for ah i}. 



{ti,. . . ,tk). 




,n-\ 
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couples the distributions of (Xi, . . . , X„) and {Yi, . . . , Y„), and A„(/?") = 1 
for any finite n. In the case where n is infinite, the proof is completed by 
applying Theorem 4.2. □ 

The next example shows that the condition in Theorem 4.3 is not nec- 
essary in general. 

Example 4.4. Let X = {^,1 — ^) and Y = (2^, 1 — C), where ^ has uni- 
form distribution on the unit interval. Then X <st Y by construction, but 
P2{xi, •) <st Q2(y2, •) only for xi > yi/2. 

4.2 Continuous-time random processes 

Denote by Di = Di{R^, Si) the space of functions from into Si that 
are right-continuous and have left limits, and equip Di with the Skorohod 
topology, which makes it Polish [9, Section 3.5]. The coordinatewise relation 
between Di and D2 is defined by 

= {{x, y) eDiX D2: x{t) ~ y{t) for all t G 

and we denote by the corresponding stochastic relation between random 
processes with paths in Di (identified as Dj-valued random elements). When 
there is no risk of confusion, the same notation shall be used for a ran- 
dom process (corresponding to and its finite-dimensional distributions 
(corresponding to R^t)- 

Lemma 4.5. R^ is a closed relation between Di and D2, whenever R is 
closed. 

Proof. Assume that Xi and x" are functions in Di such that — > Xi as 
n 00, and (x", X2) € R^ for all n. Denote A = Ai U A2, where Aj is the 
set of points t € M+ where Xi is discontinuous. It is well-known [9, Section 
3.5] that Ai is countable, and that x"(t) Xi{t) in Si for all t ^ A?. Hence 
xi{t) ~ X2{t) for all t ^ A, because R is closed. 

Observe next that if t G A, then there exists a sequence t^ £ {t, 00) n A'^ 
such that tfc — > t. Then Xi{t].) — s- Xj(t) by the right-continuity of Xj, and 
again the fact that R is closed implies xi{t) ~ X2{t). □ 

Theorem 4.6. Two random processes X and Y with paths in Di{Kj^^ Si) 
and 1)2(1^+1 5*2); respectively, satisfy X ~st Y if and only if {Xt^ , • • • , Xt^ ) ^■st 
{Ytx ) • • • 1 ^,1) for all finite parameter combinations (ti, . . . , t„). 
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Proof. The necessity is obvious. To prove the converse, define for each 
positive integer m the discretization map vr™ : Di S'"^ by 



C («)(*) 



and the corresponding interpolation map r/^ : S*™ — > Di by 

Ofc, t G [/c/m, (/c + l)/m), = 0, 1, . . . , — 1, 
^ a^2, t G (i,oo). 

The functions nf^ and 77^ are measurable with respect to the Borel a- 
algebras on A and [9, Proposition 3.7.1]. Let {X^,X^) be a cou- 

phng of 7r5"(Xi) and 7r^(X2) such that {X^,X^) G almost surely. 

Then (77^ (Xf ), ?7^(X^)) couples ??f ovrf (Xi) and r]^ o irip {X2) , and more- 
over, G i?^ almost surely. Hence 

Because rff^ o 7r["(xj) converges to x, in as jtt, ^ 00 for all Xj G ([9, 
Problem 3.12], [1, Lemma 3]), it follows that r/™o7r["(Xj) ^ Xj. Lemmas 3.9 
and 4.5 show that i?^ is closed in the weak topology, so that Xi ~st ^2- D 



4.3 Markov processes 

In the sequel, the notation X{fi, t) refers to the state of a Markov process X 
at time t with initial distribution ^, and we shall use X(x, t) as shorthand 
for X{6x,t). Markov processes Xi and X2 are said to stochastically preserve 
a relation R, if for all t, 

Xi ~ X2 =^ Xi{xi,t) r^st X2{x2,t), 

or equivalently (see Theorem 3.1), 

fJ-l ~st fJ'2 =^ -'^ll/^l,*) ~st X2{fJ,2,t)- 

The following theorem presents a simple but powerful result, which together 
with the subrelation algorithm (see Theorem 4.9 below) provides a method 
for stochastically relating (potentially unknown) stationary distributions of 
Markov processes based on their generators. 

Theorem 4.7. Let Xi and X2 be Markov processes with stationary distri- 
butions /ii and /i2 such that Xi{xi,t) —>■ fii as t ^ 00 for all initial states 
Xi. Given any measurable relation R, a sufficient condition for fii ~st f^2 is 
that Xi and X2 stochastically preserve some nontrivial closed subrelation of 
R. 



15 



Proof. Choose a pair of initial states {xi,X2) G R' , where R' is a closed 
subrelation of R stochastically preserved by Xi and X2. Then Xi{xi,t) 
and X2{x2,t) are stochastically related with respect to R' for all t, so by 
Lemma 3.9 we see that {^1,112) G R'st- Because R'^^ C Rst, it follows that 
(/Xi,/i2) G -Rst- □ 

Let Xi and X2 be discrete-time Markov processes with transition prob- 
ability kernels Pi and P2, respectively. The following result characterizes 
precisely when Xi and X2 stochastically preserve a relation R. A Markov 
process X taking values in 5i x ^2 is called a Markovian coupling of Xi and 
X2, if X{x,t) couples Xi{xi,t) and X2{x2,t) for all t and all x = {xi,X2)- 
A measurable set B is called invariant for a Markov process X, if x E B 
implies X{x,t) € B for all t almost surely. 

Theorem 4.8. The following are equivalent: 

(i) Xi and X2 stochastically preserve the relation R. 

(a) Pi{xi,B) < P2{x2, B'^) for all xi ~ X2 and compact B C Si. 

(Hi) Pi and P2 stochastically preserve the relation R. 

(iv) There is a Markovian coupling of Xi and X2 for which R is invariant. 

Proof. The implications (iv) ==?■ (i) ==?■ (iii) are direct consequences of 
the definitions, while (ii) <^=^ (iii) follows by Theorem 3.1. For (iii) =^ 
(iv), observe that Theorem 3.3 implies the existence of a coupling P of the 
probability kernels Pi and P2 such that P{x, R) = \ for all x € P. Let 
X be a discrete-time Markov process with transition probability kernel P. 
Induction then shows that X is a Markovian coupling of Xi and X2 for 
which P is invariant. □ 

Theorems 3.8 and 4.8 yield the following characterization for subrela- 
tions of a closed relation P that are stochastically preserved by discrete-time 
Markov processes Xi and X2 with continuous transition probability kernels 
Pi and P2. Denote by P* the output (10) of the subrelation algorithm in 
Section 3.2. 

Theorem 4.9. P* is the maximal closed subrelation of R that is stochasti- 
cally preserved by Xi and X2. Especially, Xi and X2 stochastically preserve 
a nontrivial closed subrelation of R if and only if R* 7^ 0. 
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Markov jump processes shall be consider next. Recall that a map Q : 
S X B[S) — > M+ is called a rate kernel on 5, if Q{x,dy) = q{x)P{x,dy) 
for some probability kernel P and a positive measurable function q. A 
rate kernel Q is called nonexplosive, if the standard construction using a 
discrete-time Markov process with transition probability kernel P generates 
a Markov jump process with paths in L)(R_|_, S) [11, Theorem 12.18]. 

Theorem 4.10 below characterizes precisely when a pair of Markov jump 
processes Xi and X2 with nonexplosive rate kernels Qi and Q2 stochastically 
preserves a closed relation R. The construction of the Markovian coupling is 
based on the local uniformization of the rate kernels Qi using the probability 
kernels Pi from 5i x 52 to Si, defined by 

n{x,B,) = ^P,{x,,B,) ^\ 6{x,,B,), (12) 

where q{x) = 1 + qi{xi) + q2{x2) [17, Section 3]. 
Theorem 4.10. The following are equivalent: 

(i) Xi and X2 stochastically preserve the relation R. 

(a) For all xi ~ X2 and compact B C Si such that 6{xi, B) = 6{x2, B^), 
Qi{xi,B) - qi{xi)6{xi,B) < Q2{x2,B^) - q2{x2)6{x2, B^). 

(Hi) The probability kernels in ( 12) satisfy Pi{x, •) ~st P2{x, ■) for all x ^ R. 

(iv) There is a Markovian coupling of Xi and X2 for which R is invariant. 

Countable spaces admit the following slightly more convenient charac- 
terization, due to the fact that all sets are measurable. 

Theorem 4.11. // the spaces Si and S2 are countable, then the properties 
of Theorem 4-10 are equivalent to requiring that for all xi ~ X2: 

Qiixi,Bi)<Q2ix2,Br) (13) 
for all Bi C Si such that xi ^ Bi and X2 ^ B^ , and 

Qiixi,B^)>Q2ix2,B2) (14) 
for all B2 C S2 such that xi ^ B^ and X2 ^ B2. 
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Remark 4.12. For order relations on countable spaces it suffices to ver- 
ify (13) for all upper sets Bi and (14) for all lower sets B2 (see Remark 2.6), 
so Theorem 4.11 becomes equivalent to Massey's characterization [20, The- 
orem 3.4]. 

To prove Theorem 4.10 we need the following special form of Theorem 3.3 
to account for the fact that Pi are probability kernels from Si x ^2 to Si, 
and not from Si to Si. 

Lemma 4.13. Let Pi be probability kernels from Si x 52 to Si such that 
Pi{x, •) ~st P2ix, •) for all X G i?. Then there exists a probability kernel P 
on Si X S2 such that P{x, •) couples Pi{x, •) and P2{x, •) for all x & SiX S2, 
and P{x, R) = 1 for all x € R. 

Proof. Denote Si = Si x S2 and S'^ = Si, i = 1,2, and define R = {{x,x) : 
x € R} and R' = R. Then by Theorem 3.3 there exists a probability kernel 
P from Si X S2 to S[ x 6*2 such that P{{x, y), •) couples Pi{x, •) and P2{y, •) 
for all X G SiX S2 and y £ SiX S2, and P{{x, x),R) = 1 for all x € i?. Define 
P{x, B) = P{{x, x),B) for all x G Si x S2 and measurable B C Si x S2. □ 

Proof of Theorem 4-10. (i) =^ (ii). Choose xi ~ X2 and a compact B C 
Si such that 6{xi,B) = 5{x2, B^). Then Theorem 2.5 shows that 

F{Xi{xi,t) G B)-6ixi,B) < P{X2ix2,t) G B"*) - 6{x2, B^) 

for all t. Dividing both sides above by t, and taking t [ 0, we thus see using 
Kolmogorov's backward equation [11, Theorem 12.25] the validity of (ii). 

(ii) =^ (iii). Observe that for any compact B C Si, 

P2{x, B--) - Pi{x, B) = q{x)-\{Q2{x2, B-) - q{x2)6{x2, B^)) 

- (Qi{xi,B) - q{xi)6{xi,B))} 
+ 5{x2,B^)-8{xi,B). 

Because 6{x2,B~*) > 6{xi,B) for all xi ~ X2, we see that Pi{x,B) < 
P2{x,B~') for all x G R. Hence using Theorem 2.5 we may conclude that 
Pi{x, •) ~st P2ix, •) for all x G i?. 

(iii) =^ (iv). Let Pi be the probability kernels defined in (12). 
Lemma 4.13 then shows the existence of a probability kernel P on 5i x S2 
such that P{x,-) couples Pi{x,-) and P2{x,-) for all x G Si x S2, and 
P{x, R) = 1 for all x G R. Define a rate kernel Q on x S2 by Q{x, B) = 
q{x)P{x,B). Then 

/ {fi{yi) - fi{xi)) Q{x,dy) = I ifiiyi) - fi{xi)) Qi{xi,dyi) 

JS1XS2 J Si 
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for all X € 5*1 X S'2 and all bounded measurable / on Si, i = 1,2, which 
implies that Q is nonexplosive (Chen [4, Theorem 37]). Let X be a Markov 
jump process generated by Q using the standard construction [11, Theorem 
12.18]. Then X{x,t) couples Xi{xi,t) and X2{x2,t) for all x € Si x S2 and 
for all t (Chen [4, Theorem 13]). Moreover, R is invariant for X, because 
P{x, R) = 1 for all x e R. 

(iv) ==^ (i). Clear by definition. □ 

Proof of Theorem 4-11- Assume first that (13) and (14) hold, and choose 
xi ~ X2 and Bi C Si so that 6{xi,Bi) = 6{x2,B^). If xi ^ Bi, and 
X2 ^ B^, then the validity of Theorem 4.10:(ii) is obvious from (13). In 
the other case where xi G Bi, and X2 € B^, denote B2 = {B^y. Then it 
follows (check) that B^ C B^, and thus xi ^ B^ . This further implies that 
X2 ^ B2, because xi ~ X2. Hence (14) shows that 

Q2ix2,B2) < Qi{xi,B^) < Qi{xi,Bl), 

from which we again see that Theorem 4.10:(ii) is valid. 

Assume next that Theorem 4.10:(iv) holds, and let xi ~ X2 be such 
that xi ^ Bi and X2 ^ . Then (13) follows by Theorem 2.5, because 
Qi{xi,Bi) = Pi{x,Bi) and Q2{x2,B^) = P2{x2,B^). Inequality (14) can 
be verified by a symmetrical argument. □ 

This section is concluded by an analogue of Theorem 4.9. A rate kernel 
Q{x, dy) = q{x)P{x, dy) such that g is a continuous function and P is a 
continuous probability kernel shall be called continuous. Given Markov jump 
processes Xi and X2 with continuous nonexplosive rate kernels Qi and Q2, 
define the relations -R*-""^ by R^'^^ = R, and 

= {x G : (A(x,-),A(x,-))G^S?}, (15) 

where Pi and P2 are given by (12). Moreover, denote R* = R^^qP^"), as m 
Section 3.2. 

Theorem 4.14. R* is the maximal closed subrelation of R that is stochasti- 
cally preserved by Xi and X2- Especially, Xi and X2 stochastically preserve 
a nontrivial closed subrelation of R if and only if R* ^ 0. 

Proof. The continuity of Qi and Q2 guarantees the continuity of Pi and 
P2. The proof is hence completed by repeating the steps in the proof of 
Theorem 3.8, with notational modifications to take into account that here 
Pi are probability kernels from Si x 52 to 5^, and not from Si to Si. □ 
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5 Applications 



5.1 Hidden Markov processes 

The goal of this section is to stochasticahy compare two hidden Markov pro- 
cesses Yi and I2 of the form Yi = fioXi, where Xi is a Markov process taking 
values in S[, and fi is a continuous function from S[ to Si. Although the 
results in this section have natural counterparts for Markov jump processes, 
we shall only treat the case where Xi are discrete-time Markov processes 
with transition probability kernels Pi. 

Theorem 5.1. Let R he a closed relation between Si and S2, and assume 
that 

Pi{xiJr\B)) < P2{X2J^\B^)) (16) 

for all xi and X2 such that fi{xi) ^ f2{x2) and all compact B C Si. Then 
Yi{t) ~st Y2{t) whenever Yi{0) ~st ^^2(0). 

The proof of Theorem 5.1 relies on the notion of induced relation, defined 
as follows. Given a closed relation R between Si and 5*2, and two continuous 
functions (j)i : 5^' — > 5j, i = 1,2, we denote 

R' = es'ixs'2: iM4),Mx2)) e R} . 

The closed relation R' is said to be induced from R by the pair ((pi, ^2), and 
we denote by Ri,^ the stochastic relation generated by R'. 

Lemma 5.2. Probability measures fii on S[ and ^2 on S2 satisfy {^1,^2) £ 
i?st 'if o.'T^d only if {m o (j)'^'^ , ^12 o (f)'^'^) G R^t. 

Proof. Observe that R' = <j)~^{R), where the function cj) is defined by 
(l){xi,X2) = {<t>i{xi),(j)2{x2)). If (/ii,/X2) € R'st, then let A be a coupling 
of /ii and fi2 such that X{R') = 1. Then A o couples the probability 
measures fii o and H2 o 4>2^ , and moreover, that A o (j)~^(R) = 1. 

Assume next that (z/i, U2) G Rst, where Ui = fiio . Choose a compact 
B' C S[, and observe that the right conjugate of B' with respect to R' equals 

[B'U, = cP2\[MB')]r), 

where [(piiB')]']^ denotes the right conjugate of (j)i{B') with respect to R. 
Moreover, because (f)i{B') is compact. Theorem 2.5 shows that 

^il{B') = i^iiMB')) < i^2{[MB')]r) = M[BV), 
and hence (fii, H2) G R'sf D 
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Proof of Theorem 5.1. Let R' be the relation between S[ and induced 
from R by the pair (/i, /2). Lemma 5.2 then shows that for aU t, 

Yi{t)4.tY2{t) if and only if 4t X2(t), 

and moreover, (16) is equivalent to requiring that the pair (i-*i,P2) stochas- 
tically preserves R'. □ 

Example 5.3 (Non-Markov processes). Let Yi and I2 be non-Markov pro- 
cesses with values in Si and 5*2, respectively. Following Whitt [29], let us 
assume that Yi can be made Markov by keeping track of additional informa- 
tion, say a random processes Zi with values in S'^. Under this assumption, 
Yi = TTi o Xi, where Xi = (Yi, Zi) is a Markov process taking values Si x S'^, 
and TTi denotes the projection from Si x S'^ onto Si. Denoting the transi- 
tion probability kernel of Xi by Pi, condition (16) in Theorem 5.1 becomes 
equivalent to 

sup Pi((yi,zi), BxS[)< inf P2((y2,^2), x S'^) (17) 

for all yi ~ 1/2 and all compact B C Si. Inequality (17) together with 
^i(O) ~st ^2(0) is thus sufficient for Yi{t) ~st Y2{t) for all t. This formulation 
is conceptually similar, though not equivalent, to [29, Theorem 1]. 

Example 5.4 (Lumpability). A Markov process X with values in 5i is 
called lumpable with respect to / : Si — > 5*2, if / o X is Markov for any 
initial distribution of X [1.3]. It is well known [(>] that X is lumpable if and 
only if its transition probability kernel satisfies 

P{x,f-HB)) = P{y,f-\B)) 

for all X and y such that f{x) = f{y) and all measurable B d S2. This is 
equivalent to saying that the pair (P, P) stochastically preserves the relation 
{{x,y) € Si X Si : f{x) = f{y)} induced by (/, /) from the equality on S2. 
Moreover, if X is lumpable with respect to /, then the pair (P, P') stochas- 
tically preserves the relation {(x,y) G 5*1 x ^2 : f{x) = y} induced from the 
equality on ^2 by the pair (/, x), where P' denotes the transition probability 
kernel of / o X, and x is the identity map on S2. The notion of lumpability 
may be generalized by calling X lumpable with respect to a relation R, if 
there exists a Markov process Y such that X and Y stochastically preserve 
R [18]. 
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Example 5.5 (Stochastic induced order). For order relations, condition (16) 
in Theorem 5.1 can be rephrased as 

Pl{xi,f{\B))<P2{x2j2\B)) 

for all upper sets B C S and all xi and X2 such that /i(xi) < f2{x2) (see 
Remark 2.6). This is a discrete-time analogue to [5, Theorem 6]. 

5.2 Population processes 

Let us denote by ei, . . . ,em the unit vectors of Z™, and define eo = and 
ejj- = —Ci + Cj for notational convenience. A Markov population process [I J] 
is a nonexplosive Markov jump process taking values^ in C Z™, generated 
by the transitions 

X ^ X + eij at rate ajj(x), i,j S {0, 1, . . . , m}, i ^ j. 

where Oij are positive functions on S such that aij{x) = for x + Cij ^ S. 
The functions ao,i and Oj^o may be regarded as the arrival and departure 
rates of individuals for colony i, and aij represents the transfer rate of 
individuals from colony i to colony j. Population processes with values in 
S C may be viewed as Markovian queueing networks [22] or interacting 
particle systems [16]. 

The following result characterizes precisely when two population pro- 
cesses stochastically preserve a relation R between S C Z™' and S' C Z"* . 
To state the result, define for x G 5" and y & S' , and for sets of index pairs 
[/ C {0, . . . , m}2 and V C {0, . . . , 

U^{x,y) = {{k,l) : X + eij ~ y + ek,i for some (i, j) G U} 

and 

V^{x, y) = : x + eij ~ y + ek,i for some {k, I) £ V} . 

Theorem 5.6. Let X and X' be population processes taking values in S C 
17^ and S' C Z™' generated by transition rate functions ai j and oi'^i, re- 
spectively. Then X and X' stochastically preserve a relation R if and only 
if for all x ~ y; 

{i,j)eu {k,i)eu^ix,y) 

^ Often it is natural to assume that the elements of S have positive coordinates, but in 
this section there is no need to make this restriction. 
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for all U C ■ x + Cij ^ y}, and 

ai,j{x)> ^ a',/y) (19) 
{i,j)eV^{x,y) {k,i)ev 

for all V C {(A;, /) : x '/^ y + ek,i}. 

Proof. Observe that for any x ~ y and B C S such that x ^ B and y ^ B^, 
the rate kernels of X and X' satisfy 

Q{x,B)= ^ ai,j{x) and Q'{y,B^)= ^ a'k,i{y), 

{i,j)&U {k,l)eU^{x,y) 

where U = ■ x + Cij G B}. Moreover, x + eij ^ y for all (i, j) e [7, be- 

cause y ^ B^ . Hence inequality (18) is equivalent to (13) in Theorem 4.11. 
By symmetry, we see that (19) is equivalent (14). □ 

Example 5.7 (Partial coordinatewise order). Define a relation between S 
and 5' by denoting x <m '^i Xi < yi for all z G M, where M is a fixed 
subset of {1, . . . ,m} fi {1, . . . ,m'}. We shall next show that X and X' 
stochastically preserve the relation <m if and only if for all x <m y and all 
k in the set Mq{x, y) = {i G M : Xi = yi}: 

i&I i&I\J([Q,m']\Mo(x,y)) 

for all / C [0,m] \ {/c}, and 

Y OLk,j{x)>Y^'kj{y) (21) 

jGJU([0,m]\Mo(x,j/)) ieJ 

for all J C [0,m'] \{A;}. 

To justify the above claim, observe that for any x <m y and U is as in 
Theorem 5.6, we can write U = Ukexilk x {^}) for some Ik C [0,M] \ {k} 
and K C Mo(x,y). Hence U-t{x,y) = UkeKi-Jk x {^})) where U-,{x,y) = 
Ukf^xih U ([0,m'] \ Mo{x,y))) x {/c}). By summing both sides of (20) over 
k G K, with Ifc in place of /, we see that (18) holds. On the other hand, it 
is easy to see that (18) implies (20), and a symmetric argument shows the 
equivalence of (19) and (21). 

The above characterization extends and sharpens earlier comparison re- 
sults for Markovian queueing networks [7, 16, 17]. For population processes 
where transfers between colonies do not occur, (20) and (21) simplify to 

ao,k{x) < OiQ kiy) and akfi{x) > a'^^Qiy) 

for all X <M y and all /c G M such that Xk = yk [-, Lemma 1]. 
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5.3 Parallel queueing system 

Consider a system of two queues in parallel, where customers arrive to queue 
k at rate S (0, 1) and have unit service rate. Assuming all interarrival 
and service times are exponential, the queue length process X = (Xi, X2) is 
a Markov population process on with transition rates ao.fc(^) = and 
Q^fc,o(^) = ^i^k > 0), k = 1,2. We shall also consider a modification of the 
system, where load is balanced by routing incoming traffic to the shortest 
queue, modeled as a Markov population process = {Xl'^,X2^) with 
transition rates 

«o,i(a^) = (^1 + A2)l(xi < X2) + Ail(xi = X2), 
«o,2(a^) = (^1 + >^2)Kxi > X2) + A2l(xi = X2), 

and ct^^(x) = a^fiix) for k = 1,2. Common sense suggests that load bal- 
ancing decreases the total number of customers in the system, so that 

Xl^it) + X^^{t) <st Xi{t) + X2{t). (22) 

However, the justification of (22) appears difficult, because using Theo- 
rem 5.6 can check that the processes X^^ and X do not stochastically 
preserve the coordinatewise order on Z^, nor the order 

i?-- = {(x,y):|x|<|y|}, 

where = xi + X2- On the other hand, it is known [30] that (22) holds 
for all t, whenever X^^{0) = -^(0), which suggests that X^^ and X might 
stochastically preserve some strict subrelation of i?^"™. The following theo- 
rem summarizes the output of the subrelation algorithm applied to the rate 
kernels of X^^ and X. 

Theorem 5.8. Starting from R^^^ = R^^^, the subrelation iteration (15) 
produces the sequence of relations 

= {(x,y) : \x\ < \y\ and xi\J X2 < yi V y2 + (yi A y2 - ri)+} , (23) 

which converges to 

R* = {(x, y) : |x| < \y\ and xi V X2 < yi V 2/2} • 

The limiting relation R* may be identified as the weak majorization 
order on Zl (Example 2.3). As a consequence, 

XL^(O) ^^"^ X(0) =^ X^^{t) ^^t"" X{t) for ah t. 
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Especially, ^^^(0) ^""^ X(0) implies (22), and moreover, 

which indicates that the queue lengths corresponding to X^^ are more bal- 
anced than those corresponding to X. The proof of Theorem 5.8 is based 
on the following lemma, the proof of which is omitted. 

Lemma 5.9. The function an{x) = xi V X2 V (|2;| — n) on 1? satisfies: 

(i) a„(x) = \x\ — xi A X2 A n. 

(a) an{x - Ck) < ctn+iix) for all k. 

(Hi) an{x + ei) > a„(x + 62) if and only if xi > 3:2 V (|x| — n). 

(iv) an{x — ei) < a„(x — 62) if and only if xi > X2 V (|x| — n — 1). 

(v) a„(x + Ck) > an{x) for k such that x^ = xi V X2. 

(vi) a„(x — efc) = a„+i(x) for k such that x^ = xi A X2. 

Proof of Theorem 5.8. Define the function an(x) as in Lemma 5.9. Then 
the relations i?^"^ defined in (23) can be written as 

= {(x,y) : |x| < \y\, a„(x) < a„(y)} , 

which shows that i?^"^ is an order for all n. To show that R^"'^ is the sequence 
of relations produced by iteration (15), it is sufficient (Remark 4.12) to show 
that the properties 

• QL^(x, U) < Q{y, U) for all i?(")-upper sets U such that x,y^U, 

• Q^^{x, U) > Q{y, U) for all /^("^-lower sets U such that x,y (^U, 

are valid for all (x, y) G R^'^'^^\ and that at least one of the above properties 
fails for (x,y) € \ 

Assume {x^y) € ii^""*"^^ (actually, i?^") is here enough), and let U be an 

R{n) 

-upper set such that x,y ^ U. Then x — e^, y — for all k. 

(i) If y + ei,y + e2 G U, then Q^^{x,U) < Ai + A2 = Q{y,U). 

(ii) Assume y + ei,y + 62 ^ C/, and choose / so that = yi V y2- Then 
Lemma 5.9:(v) together with an{x) < an{y) imply that q;,„(x + e^) < 
a„(x) -|- 1 < an(y + ez) for all fe. Hence x + ^ t/ for all fc, because 
?7 is i?(")-upper, so that Q(x, U) = = Q^^iy, U). 
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(iii) Assume y + ei £ U,y + €2 ^ U. Then a„(y + ei) > a„(y + 62), so 
Lemma 5.9:(iii) shows that yi > y2 and yi > \y\ —n. Now if x + e^ S U, 
then a„(x + efc) > an{y + 62) = Oin{y)- Further an(2;) < cxn{y) shows 
that a„(x) = Oniy), so that an{x) = yi > \x\—n. Because Q;„(x+efc) > 
an{x) > \x\ — n, it follows that Xk = xi V X2 = yi- As a consequence, 
|x| < \y\ implies that xi A X2 < y2 < yi, so that > xi A X2- Hence 
a]f{x) = 0, which implies that Q^^{x,U) = < Q{y,U). The case 
where y + ei ^ U,y + ei £ U is similar. 

Assume next that {x,y) G and let U be an i2^")-lower set such 

that x,y ^ U. Then x + e^, y + ^ [/ for all k. 

(i) Assume y-ei € C/, ?/-e2 ^ U. Then 2/2 = or a„(y-ei) < an{y-e2), 
so with the help of Lemma 5.9:(iv), we see that ?/i > 2/2 V (|y| — n — 1). 
Choose k so that x^ = xi V X2, and observe that 

{x,y — ei) ^ i?^") implies that either \x\ = \y\ or a„(x) > an{y), so 
that Xfc > 0. If xi 7^ X2, then an(x — e^) = an(x) — 1 < an{y) — 1 = 
"n(?/ — ei). If xi = X2, then |x| < \y\ together with yi > y2 implies that 
xi < yi. Hence an(x-efc) = xiV(|x|-7i-l) < (yi - 1) V (|?/| -n- 1) = 
<^n{y — si)- Thus, we may conclude that x — G C/, which shows that 
Q^^{x,U) > 1 = Q{y,U). By symmetry, the same conclusion holds 
under the assumption y — ei^U,y — 62^11. 

(ii) Assume y — ei,y — 62 € U, and choose I so that yi = y\ A 7/2- Then 
using Lemma 5.9:(ii) and Lemma 5.9:(vi) we find that for all /c, 

a„(x - Cfc) < an+i{x) < an+i{y) = a„(y - ei). 

Further, if |x| = \y\, then Lemma 5.9:(i) together with xi V X2 < 
a„+i(x) < a„+i(y) shows that xi Ax2 = |x| — xi Vx2 > yi A?/2 A 1), 
so that xi Ax2 > 1- On the other hand, if |x| < \y\, then x ^ U implies 
that an{x) > an{y — ei) > an+i{x), so it follows that xi Vx2 < |x| — n, 
which again shows that xi A X2 > 1. Hence x — € U for all k, and 
we may conclude that Q^^{x, U) = 2 = Q{y, U). 

(iii) Assume y - q ^ C/ for all I. Then Q^^(x, U) > = Q{y, U). 
Finally, assume that {x,y) G i?^'") \ Then 

Q„+i(y) < a„+i(x) < a„(x) < a„(y), 

which implies that yi V 7/2 < |y| — n and a„(x) = an{y) = \y\ — n. Consider 
the i?(")-lower set U = {z : \z\ < |y|, an{z) < ««(?/)}• Then x,y ^ U, 
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X -\- ek,y + Gk ^ U for all k, and y — ei G [/ for all I. On the other hand, 
(Xnix) = \y\ — n together with |x| < \y\ shows that an{x) = xi V X2, so 
that a„(rc — e^) = a„(x) for some k. Especially, x — Ck ^ U, so that 
< 1<2 = Q(?/,C/). □ 

6 Conclusion 

This paper presented a systematic study of stochastic relations, which nat- 
urally extend the notion of stochastic orders to relations between random 
variables and processes that may take values in different state spaces. The 
key points of the paper may be summarized by Theorem 3.8, which char- 
acterizes the existence of subrelations stochastically preserved by a pair of 
probability kernels, and Theorem 4.7, which underlines the relevance of sub- 
relation techniques in stochastically comparing stationary distributions of 
Markov processes. Finite-state Markov processes and diffusions are two im- 
portant classes of processes that were not discussed in the paper. In finite 
state spaces the subrelation algorithm converges in finite time, which calls 
for numerical analysis of the runtime. The analysis of stochastic relations 
for diffusion processes requires the identification of suitable test functions 
that behave well with respect to taking relational conjugates. These issues 
may be considered interesting topics for future research. 
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